Beyond the Bar Graph

• Data use is increasing rapidly within the education space
• Policymakers are under increasing pressure to use data to inform decisions, justify funding, and guide practice
• But, policymakers are often not statisticians, researchers, or quants
• Data visualization is a way to bridge this gap
• Proper data visualization will bring the data to the audience in a way they can understand quickly and use to inform decisions

Data visualization is a tool for communicating a specific feature of a datset in an approachable and efficient manner

If a picture is worth a thousand words, a good data visualization must always be better than a table.

1. Review data visualization principles
2. Look at applications in education data
3. Best practices and advice
4. Review tools to use
5. Activity!

How can we improve this simple scatterplot?

There are a few things that all charts need [most of the time!]:

• Axis labels and a title
  • These make the chart self-explanatory
• A legend
  • What is the unit in the graphic?
• A scale
  • How are units mapped to the visual space
• Annotations
  • Author and data source (depending on distribution)

These add up to provide context and meaning to viewers who have not analyzed the dataset themselves.

• Charts and data are made up of dimensions (e.g. a bar chart is x and y)
• Additional dimensions can be represented by additional aesthetics or chart elements (e.g. color, size, shape, etc.)
• Dimensions can also be shown by multiple plots (e.g. a filmstrip)
• Smart use of dimensions allows us to increase the information density of our charts

How you turn dimensions in the data into visual cues for your audience is everything.

• Any given dimension may be measured at different levels of measure [derived by Stanley Smith Stevens in the 1940s and 50s]
  • Nominal: unordered categories of data (e.g. race)
  • Ordinal: ordered categories of data, relative size and degree of difference between categories is unknown (e.g. Likert scales, proficiency levels, etc.)
  • Interval: ordered categories of data, fixed width (e.g. grade level in school, GPA)
  • Continuous (ratio): a measurement scale in a continuous space with a meaningful zero (e.g. scale scores)

Aesthetics for Mapping

How do we map levels of measurement onto visual features of charts?

Ordered vs. Unordered

• Focus on the content and the context
• Know your audience, understand their limits, then push them!
• Use best practices, except when they don't help
• Experiment and iterate!

## stat_bin: binwidth defaulted to range/30. Use 'binwidth = x' to adjust this.

Think like a map. Data density and easy interpretability.

How do we display thousands or hundreds of thousands of observations with varying data types across multiple dimensions?

1. Summarize the data
   • Display summary statistics visually depicting the central tendency and spread of data
2. Plot the raw data
   • Annotate wisely to display the main message
3. Model the data
   • Use a statistical model to summarize features of the data

Let's look at some examples of this.

• The most simple summaries are measures of central tendency, most easily understood
• It is important to look at the spread of data too though
• If time is of interest, we are interested in trends
• If space is of interest, we are interested in maps or spatial distributions
• Think about context and reference
• Let's look at an example summarizing student data to schools!

Here is a simple plot of mean school reading scores:

But, what's wrong with this plot?

We still aren't sure what the mean scale score means. Let's see a couple more additions that can make this useful.

Sometimes, we can get away with showing the raw data, that is, all data points. We may want to do this for a few reasons:

• the "wow" effect,
• because it is easier,
• or because it looks better aesthetically.

How could it be done?

• Without reducing the data points we need to do one or more of three things to be successful

1. Spread the data out
2. Bin the data into groups
3. Annotate and add references

1. Repeat the plot for subgroups
   • Allow the viewer to draw comparisons across small plots
2. Reduce the ink
   • Each point has too much "weight", so make them weigh less
3. Add Reference Points
   • 600,000 observations in one panel is not meaningful and overwhelms the viewer

• Edward Tufte and others recommend small multiples, a technique of repeating a plot across groups to compare relationships in multiple dimensions

## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.

## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.

## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.
## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.

All models are wrong. Some models are useful.

## geom_smooth: method="auto" and size of largest group is >=1000, so using gam with formula: y ~ s(x, bs = "cs"). Use 'method = x' to change the smoothing method.

We can combine these features.

• Facets with smoother lines for references (small multiples + models)
• Summary plots with raw data in the background
• Reference lines and group comparisons

• Annotation
• Labeling
• Lots of data-ink
• Reference points galore

• Have a properly chosen format and design
• Use words, numbers and drawing together
• Reflect a balance, a proportion, relevant scale
• Display an accessible complexity of details
• Have a narrative quality, tell a story
• Avoid content-free decoration (Tufte's proverbial chartjunk)
• Draw in a professional manner with an eye on the technical details
• Provide your audience with the context they need and remember they haven't lived in the data like you!

They can communicate, confound, brand, and distract

The technology you choose to do visualizations is largely a question of personal productivity, but with some important caveats:

• In the future, more and more content is going to be delivered in a paperless world, so pick a technology that can leverage web/tablet/phone interfaces
• Different technologies are useful for different levels of finish and polish; Adobe Illustrator is great for publication quality graphics, R is a great tool for rapidly prototyping different visualizations
• Chose a technology that best serves your consumer, not you the producer. Charts are a service to the consumer, not to the creator.

Keep in mind that depending on the project you may need to programatically make data visualizations, or you may need a highly customized illustrated graphic.

We have a number of other techniques we can use beyond simple charts.

• Animations
• Interactive demos
• Summary tables
• Videos
• Web sites
• Simulations

• Tufte, Edward. 1992. The Visual Display of Quantitative Information. Graphics Press.
• Unwin, Theus, and Hofmann. 2006. Graphics of Large Datasets: Visualizing a Million. Springer.
• Wilkinson, Leland. 2005. The Grammar of Graphics. Springer.
• Ware, Colin. 2012. Information Visualization: Perception for Design. 3rd Edition. Morgan Kaufmann.
• Cleveland, William. 1994. The Elements of Graphing Data. Hobart Press.
• Cleveland, William. 1993. Visualizing Data. Hobart Press.

• Few, Stephen. 2009. Now You See It: Simple Visualization Techniques for Quantiative Analysis. Analytics Press.
• Few, Stephen. 2012. Show Me the Numbers: Designing Tables and Graphs to Enlighten. Analytics Press.
• Yau, Nathan. 2011. Visualize This: The FlowingData Guide to Design, Visualization, and Statistics. Wiley.
• Few, Stephen. 2006. Information Dashboard Design: The Effective Visual Communication of Data. O'Reilly Media

• Andy Kirk
• D3
• The Functional Art
• ggplot2
• Robert Kosara
• Data Stories Podcast

• DEWS Homepage: http://wise.dpi.wi.gov/wisedash_dews
• E-mail: jared.knowles@dpi.wi.gov / jknowles@gmail.com
• GitHub: http://www.github.com/jknowles
• Homepage: www.jaredknowles.com
• Google+: https://plus.google.com/+JaredKnowles

print(sessionInfo(),locale=FALSE)

## R version 3.1.0 (2014-04-10)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## 
## attached base packages:
## [1] grid      stats     graphics  grDevices utils     datasets  methods  
## [8] base     
## 
## other attached packages:
##  [1] vcd_1.3-2              plyr_1.8.1             mgcv_1.8-3            
##  [4] nlme_3.1-118           hexbin_1.26.3          lattice_0.20-29       
##  [7] ggthemes_1.9.0         eeptools_0.3.1         MASS_7.3-35           
## [10] ggplot2_1.0.0          knitr_1.7.1            slidifyLibraries_0.3.1
## [13] markdown_0.7.5         whisker_0.4            slidify_0.4.5         
## [16] devtools_1.6.1        
## 
## loaded via a namespace (and not attached):
##  [1] abind_1.4-0      arm_1.7-07       car_2.0-21       chron_2.3-45    
##  [5] coda_0.16-1      codetools_0.2-9  colorspace_1.2-4 data.table_1.9.4
##  [9] digest_0.6.4     evaluate_0.5.5   foreign_0.8-61   formatR_1.0     
## [13] gtable_0.1.2     httr_0.5         labeling_0.3     lme4_1.1-7      
## [17] maptools_0.8-30  Matrix_1.1-4     memisc_0.96-10   mime_0.2        
## [21] minqa_1.2.4      munsell_0.4.2    nloptr_1.0.4     nnet_7.3-8      
## [25] packrat_0.4.1-1  proto_0.3-10     Rcpp_0.11.3      RCurl_1.95-4.3  
## [29] reshape2_1.4     scales_0.2.4     sp_1.0-15        splines_3.1.0   
## [33] stringr_0.6.2    tools_3.1.0      yaml_2.1.13